Add Wallet Integration Tests and Fix Descriptor Persistence Issues#821
Add Wallet Integration Tests and Fix Descriptor Persistence Issues#821moisesPompilio wants to merge 9 commits intogetfloresta:masterfrom
Conversation
|
The first commit seems unrelated |
The first commit introduces |
jaoleal
left a comment
There was a problem hiding this comment.
Concept ACK.
Heres some initial review
| self.run_node(self.florestad) | ||
|
|
||
| self.log("Checking initial descriptors...") | ||
| descriptors = self.florestad.rpc.list_descriptors() |
There was a problem hiding this comment.
One thing to add, both listdescriptors.py and loaddescriptor.oy is fine but they are dependent on each other, if one fails they will both fail...
that will be a problem if breaking one raises a false alarm about the other.
There was a problem hiding this comment.
In integration tests it's common to have some interdependencies, such as between wallet_conf and wallet_flag. They also depend on listdescriptors, because that's the single method we use to check the descriptors Floresta has stored.
There was a problem hiding this comment.
Yeah, fine to depend on listdescriptors, but listdescriptors test dont need to depend on loaddescriptor you can instantiate and insert a descriptor from the Config.toml. It can help for this case.
There was a problem hiding this comment.
But in this case, if there is a problem with listdescriptors, the wallet_conf and wallet_flag tests will also fail, because they also depend on that RPC.
| /// Parses each descriptor, validates it, and derives the specified number of addresses | ||
| /// starting from the given index. | ||
| pub fn derive_addresses_from_list_descriptors( | ||
| descriptors: &[String], |
There was a problem hiding this comment.
There was a problem hiding this comment.
The descriptor is already a String, so using a &[&str] will make us waste resources converting it to &str. It's better to use it as a String directly; this method is used inside the wallet internals which already obtain the descriptor as a String.
There was a problem hiding this comment.
Theres no cost casting to a &str. but i understand thats how our codebase is written right now, fine.
IIUC theres no cost at all to cast Strings to &strs, but theres some when casting &str to String, basically the same relation between [B] and Vec<B>... Being picky with this may lead us to less runtime allocs.
The ideal is to declare functions that only read and compare strings with a generic S where S: AsRef<str>
There was a problem hiding this comment.
The problem here is that we can have multiple database implementations; so if we pass only pointers (&) to objects retrieved from the database it will be wrong, because the objects live shorter than the pointers. It's better to pass the object in this case, for example we pass a Vec<String> for descriptors, because if we passed Vec<&str> the &str would need to live longer than the Vec<String> that only exists inside the descriptor-retrieval function.
That's why when we receive a Vec<String> and it needs to be used by another function, that function takes a &[String] to avoid allocating memory by creating a new array of strings.
|
Needs rebase |
27e55ad to
c2bff7d
Compare
|
c2bff7d Performed a rebase and added some review requests. |
|
In ae433c1 there's still a |
c2bff7d to
5ce80a0
Compare
7b9921f to
7f9b79b
Compare
|
7f9b79b I did the rebase. |
| fn parse_xpubs(xpubs: &[String]) -> Result<Vec<Descriptor<DescriptorPublicKey>>, FlorestadError> { | ||
| fn parse_xpubs( | ||
| xpubs: &[String], | ||
| network: Network, |
There was a problem hiding this comment.
I think you dont need to take ownership to read enums
There was a problem hiding this comment.
I couldn't understand the comment here
There was a problem hiding this comment.
You can read the network enum by reference
There was a problem hiding this comment.
Copy types should never be passed by reference
Created a `constants.py` file to centralize all constants used in the test framework. This change improves maintainability by providing a single location for managing constants and makes it easier to understand their purpose and usage across the tests.
Updated the documentation for the `wallet_descriptor` option to explicitly describe its behavior with descriptors. The previous text incorrectly referred to xpubs, which could cause confusion.
Previously, it was possible to save the same descriptor multiple times, leading to redundancy and potential inconsistencies. Adds a verification step in the `push_descriptor` method to ensure that a descriptor is only added if it does not already exist.
- Updated xpub parsing to reject xpriv and xpubs related to multisig, ensuring only standard public keys are accepted for descriptor generation. - Fixed an issue in descriptor generation by xpub where only `wpkh` descriptors were being created. Descriptors are now generated dynamically as `wpkh`, `pkh`, or `sh-wpkh`, depending on the xpub provided. - Added a network validation step to ensure the xpub matches the network the node is running on. - Added \_typos\.toml configuration to prevent https://github.com/crate-ci/typos from analyzing words with lengths between 32 and 150 characters, as these correspond to hashes, public keys, private keys, XPUBs, descriptors, and Bitcoin addresses.
- Added `wallet_conf.py` to test wallet loading via `config.toml`. This test verifies: - Descriptors and XPUBs are correctly loaded from the configuration file. - XPRIVs and descriptors with private keys are rejected. - Added `wallet_flag.py` to test wallet loading via command-line flags. This test performs the same checks as `wallet_conf.py`, but uses flags to pass wallet information during node initialization.
7f9b79b to
4e34539
Compare
4e34539 to
e18de15
Compare
- Added tests to verify that the `loaddescriptor` RPC method can successfully load a descriptor. - Ensured that duplicate descriptors cannot be loaded. - Verified that descriptors containing private keys are rejected.
…thod - Added tests to verify that the `listdescriptors` RPC method correctly displays the descriptors loaded in the node. - Ensured that the method returns all loaded descriptors in the expected format.
e18de15 to
1c97cb5
Compare
…-only` module Improved code organization by consolidating descriptor handling in a single module - Moved `parse_descriptors`, `parse_xpubs`, and `slip132` to the `floresta-watch-only` module. - Centralized descriptor and XPUB parsing logic, previously scattered across `floresta-node` and `floresta-common`. - Removed descriptor-related features and the `miniscript` dependency from `floresta-common`.
… and refactor wallet handling
- Centralized address derivation in `floresta-watch-only`:
- Address derivation is now automatically performed when pushing a descriptor.
- Added `push_xpub` method to convert `XPUBs` into `descriptors`, push them,
and derive addresses for the wallet.
- Removed `walletInput` file from floresta-node:
- This file was no longer necessary as its functionality (`XPUB`, `descriptor`,
and address derivation) is now handled automatically by `floresta-watch-only`.
1c97cb5 to
21eca0d
Compare
Description and Notes
Adds comprehensive integration tests for wallet functionality and fixes several issues related to descriptor persistence and handling.
Key fixes:
Duplicate descriptor prevention: Added centralized validation to prevent the same descriptor from being stored multiple times in the database. The
push_descriptormethod now checks if a descriptor already exists before adding it.Improved xpub parsing and descriptor generation:
key improvements:
Centralized address derivation: Address derivation logic has been moved to
floresta-watch-only. When a descriptor is added, addresses are automatically derived and cached, simplifying wallet initialization and management.Consolidated descriptor parsing: Moved
parse_descriptors,parse_xpubs, and slip132 logic to thefloresta-watch-onlymodule, removing scattered code fromfloresta-nodeandfloresta-common. Removed the miniscript dependency fromfloresta-common.Documentation clarity: Updated wallet_descriptor option documentation to accurately describe descriptor behavior (previously incorrectly referred to xpubs).
Typos configuration: Added
_typos.tomlto prevent the typos checker from analyzing strings between 32-150 characters (hashes, keys, addresses, descriptors).New integration tests:
wallet_conf.py: Tests wallet loading viaconfig.toml, verifying correct descriptor/xpub loading and rejection of xprivs and private key descriptorswallet_flag.py: Tests wallet loading via command-line flags with the same validationsRPC method tests for
loaddescriptor(including duplicate detection and private key rejection) andlistdescriptorsConstants centralization: Created
constants.pyto centralize all constants used across integration tests, improving maintainability.How to verify the changes you have done?
wallet_conf.py,wallet_flag.py,loaddescriptor,listdescriptors) pass successfully.loaddescriptorRPC method.